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ABSTFACT 

This report describes the third study in a program of 
research dealing with the relationships between the characteristics 
of human tasks and the abilities required for task performance. The 
goal of the program is zo generate principles which can be used to 
identify ability requirements from knowledge of the characteristics 
of a task and of variations in the conditions of task ' performance. 
The study investigated the relatibnship between variations in a' 
prototypic problem-solving task^ concept identif ication^ and 
consequent changes in the abilities related to problem-solving 
performance. Characteristics of the problem-solving task were 
manipulated by varying the formal difficulty and perceptual 
complexity of the problems. Subjects' performed the criterion task 
under the different experimental conditions^ and then received a 
battery of reference tests designed to measure abilities which were 
hypothesized to relate to problem-solving performance. The test 
battery was factor analyzed to identify a reference ability 
structure. The loadings of the various criterion task conditions on 
that s^^ructuie were then estima,ted. Pesults suggested that certain 
task variations change the nature of the task in such a way that 
subjects change th^fir approach or strategy for dealing with the task. 
Such changes may require different ability profiles; thus they roay^ 
account for changes in abilities related to performance as a function 
of changes in task characteristics. Further analysis is planned to 
examine the 'interactions of task variation, subject strategics, and 
ability profiles. (Author/EJG) 
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in the abilities, related to problem-solving performance. Characteristics of 
the problem-solving task were manipulated by varying the formal difficulty and , 
perceptual complexity of the problems. Subjects performed the criterion task 
under the different experimental conditions, and then received a battery of 
reference, tests designed to measure abilities which were hypothesized to relate 
to problem-solving performance. To determine the relationship between task 
characteristics and ability requirements, the test battery was factor analyzed 
to identify a reference ability structure. The loadings of the various crite- 
rion task conditions on that structure were then estimated. 

Six separate ability factors were identified; four were found to be re- 
lated to criterion task performance. The ability relationships seemed not 
only to reflect task variations, but also to depend on the various dependent 
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The results of che study suggested that certain task variations change the 
nature of the task i/i such a way that subjects change their approach or strat- 
egy for dealing with the task- Such changes in approach may require different 
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INTRODUCTION 



The ability to make accurate personnel decisions has become increasingly 
important in the modern Navy. The imposition of severe budget constraints, 
the creation of a volunteer service, and the introduction of new systems, 
jobs, and tasks have all contributed to the need to maximize the efficient 
utilization of manpower. For the past several years the American Institutes 
for Research has been involved in studying an important aspect of this 
'problem: the development of systems for the description and classification 
of tasks which would permit more effective selection, placement, and 
training of personnel. Support for this work has come from many sources, 
including ARPA, NTEC, and, currently, ONR, 

Research by Fleishman anci his associates focused on the development of 
several systems or taxonomies for the description and classification of 
tasks (e.g., Fleishman, Kinkade, & Chambers, 1968; Fleishman & Stevenson, 
1970; Fleishman, Teichner & Stephenson, 1970; Theologus, Romashko, & 
Fleishman, 1970; Wheaton, Mirabella, & Farina, 1971; Levine^, Romashko, & 
Fleishman, 1971; Farina & Wheaton, 1971; Wheaton & Mirabella, 1972), The 
fundamental hypothesis underlying this research was thatj tasks might usefully 
be described not only in terms of more physically descriptive task taxonomies* 
focusing on display, control, a^nd procedural dimensions, but also in terms 
of the abilities necessary to perform them. Given such an abilities descrip- 
tion of a task, the personnel decision maker would be capable of selecting, 
placing, or determining appropriate training for any individual by comparing 
that individual\*s abilities with those required by any job in question, 

A second and related hypothesis which has emerged from this work is 
that there may exist a set of principles which could be used to relate the 
objective, physical characteristics of a task to task demands or the 
abilities required by the task. For example, Fleishman (1957) demonstrated 
that in a choice reaction time task where stimulus and response locations 
were intially in a simple spatial correspondence, individual differences 
were accounted for primarily as a function of the factor-analytically defined 
Perceptual Speed ability. However, as spatial correspondence of the 
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stimuli and response decreased (the display, surface was rotated relative 
to the response surface), two other factors became important—Spatial 
Orientation and Response Orientation. This finding suggested the 
possibility of translating between the two.klnds of task descriptive 
languages: that which is based on abil ity requirements , and that which 
describes tasks in terms of their physical characteristics . / 

By demonstrating that, as a physical task characteristic was manipvlatedr: 
abilities could play a changing role in performance, the study suggested 
the need for further research on how tasks, having scalable physical differences 
might be described in terms of task demands or' abil ity_rec^uirements. Based 
on such research, physical changes which occur as a task characteristic is 
varied would be translated into new ability requirements appropriate to the .. 
new version of the task. 

A system for translation of physical task dimension changes into , : 
ability requirement changes must be capable of dealing with the many kinds 
of relationships which are possible, since each pattern may have different 
personnel implications. 'For example, the correlation between a given 
ability and performance may remain constant as the task characteristic in 
question is varied. This would suggest that no change would be necessary 
in selection instruments or training programs for a newly developed system 
which includes changes along this particular characteristic, for this given 
ab'ility. 

.A second possible pattern could occur if an ability-performance 
correlation increased as a task characteristic was systematically changed. 
For example,, in^the 1957 Fleishman study cited above, the correlation between 
Spatial Orientation and task performance increased as disp^lay rotation 
increased, whiji the .correlation between Perceptual Speed and criterion task 
performance^^eccfiaied. In this case one might want either to screen 
candidate/ mo re stringently for their abilities in Spatial Orientation or 
provide special training for responding to display rotations. Other possible 
task dimension-ability patterns ^clude decreasing correlations, and non- 
monotonic trends, each with its own personnel processing implications. The 
full (and often complicated) picture emerges when all of the required 
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abilities are considered simultaneously, and multiple task characteristics 
are varied. 

The study by Fleishman (1957) and another by Zimmerman (1954), using 
the experimental -correlational method, each varied a single task characteris- 
tic, Wheaton, Shaffer (Eisner), Mirabella, & Fleishman (1973) used the 
same method to examine task demands in an auditory signal identification 
task. Unlike the earlier studies, however, two task cnaracteristics (signal 
duration and signal -to-noise ratio) were experimentally m using 
a factorial design. Ifi line with the overall goal of fi; zu^^ charac- 
teristics which related reliably to task demands or abilit-ies, it was hoped 
in this study that each task characteristic would generate its own set of 
task demand patterns. Instead, the findings indicated that changes in 
either ci ^^^acteristic produced similar changes in task demands. Correlations 
of the Auditory Perception factor with performance increased as either 
(or both) task characteristic made the task more difficult, while four 
other ability factor-performance correlations were small and remained 
constant as the task characteristics were ch'nged. 

Rose, Fingerman, Wheaton, Eisner, & Kramer (1974) extended the use of 
the experimental-correlational method to the cognitive area, using electronic 
fault-finding as the criterion task., The characteristic "formal difficulty" 
was manipulated by varying the number of possible faults; "perceptual 
complexity" was manipulated by varying the layout of components in the circuit, 
while the circuits themselves remained functionally isomorphic. A reference 
battery of 21 tests was assembled to collect data on five cognitive ability 
factors which were hypothesized to relate to performance on the fault- 
finding task. Four of these factors were correlated with performance on the 
criterion task. One of these (Syllogistic Reasoning) had a fairly constant 
relationship to criterion performance across the task manipulations, while 
another (Flexibility of Closure/Spatial Scanning) increased in importance 
as both formal difficulty and perceptual complexity inrreased. The re- 
maining two factors (Induction and Associative Memory) decreased in importance 
as perceptual complexity increased, but remained constant across all levels 
of formal difficulty. 

3 

ERiC 13 



Thus, in this'study, three different patterns of task demand-task 
characteristic patterns were observed: one factor had a constant correla- 
tion with perfonnance, unaffected by the task characteristics manipulated; 
a iecond had Increasing loadings as both characteristics were varied; 
two other- ocr casing loadings a? one characteristic changed, while 
their loa ^, 'vere unaffected by changes in the second characteristic. 
Finally, one additional and important result was obtained:; the task 
characteristic-ability relationship interacted with the dipendent measure 
examined. Five such measures were used and a three-way interaction among 
ability factors, task characteristics, and measure of performance was 
uncovered. This interaction was interpreted in terms of a strategy model 
of perfornance. The authors suggested several strategies which, if adopted 
by subjects of varying ability levels, could produce the findings they 
obtained. 

Because of the provocative outcomes in the previous study, investigation 
of another cognitive task was undertaken. Concept identification was 
selected because it is a prototypic problem-solving task, which has been 
extensively investigated in the eAperimental laboratory (e.g.. Bourne, 
1966; Bourne. Ekstrand, S Dominowski, 1971). In concept identification- 
tasks, subjects must determine the basis for classification of a series 
of stimuli. Thus it corresponds to such Navy tasks as aircraft or ship 
identification, where a large number of targets must be classified based 
on attributes such as track on a radar display, visual silhouette, or 
sound. In the present study, two classes of visual stimuli were used, and 
(he classification was based on a single attribute. Two task characteristics, 
formal difficulty and perceptual complexity, were varied to provide a 
correspondence between this study and the fault-finding experiment described 
above. A modified version of the cognitive reference battery developed 
in the previous study was used to obtain data on seven ability factors 
hypothesized to relate to performance in the concept identification task. 
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METHOD 

Approach 

The general approach was similar to that used in the previous studies 
in this series (Wheaton, et al., 1973; Rose, et aK , 1974). The study 
was conducted usjng an experimental -correlational method, in which a chosen 
criterion task was experimentally manipulated along selected physical 
dimensions in a within-subjects design. Each subject performed under all 
versions of the task. In addition, all subjects were administered a 
reference battery designed to provide (through factor analysis) a description 
of their abilities. Subjects' ability scores were then correlated with 
performance on each version of the criterion task. The result was a table 
of correlations between abilities and performance at each level of any speci- 
fied task dimension. 

Subjects 

The subjects employed in this study were 128 college students (59 males 
and 69 fenales) recruited from the American University via an advertisement 
in the university newspaper. They were paid $20.00 for their participation 
upon completion of a single day (approximately 6.5 hours) of testing. 

Experimental Criterion Task 

In the current research effort, a concept-identification task was 
employed as the criterion task. Concept identification can be defined 
broadly as the ability to discern "regularity in real or imagined events 
or objects" and to employ instances of this regularity in a manner appro- 
priate to the circumstances (Bourne, et al . , 1971). In laboratory research 
studies it frequently involves the formation and testing of hypotheses 
in a problem-solving situation in order to identify correctly a classification 
rule or principle. In the current study subjects were presented with a 
sequence of five pairs of faces in which one facial feature had been 
designated as the solution to the problem. For each pair, subject^ were 
to guess which face had the correct feature. After responding th/y were 
told which face contained the relevant feature, and were then presented with 
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the next stimulus pair. Their task was to infer which feature was common 
to all the '^correct" faces and was, therefore, the solution to the problem. 

Stimuli . The stimuli were facial composites constructed from a police 
identification kit manufactured by the IdenLiKit Company (1960, Bangor 
Punta Operations, Inc.)- The kit contained a variety of individual facial 
characteristics, including such specific features as eyes, eyebrows, noses, 
chins, ears, lips, and hair, which could serve as stimulus dimensions. The 
kit also offered many potential values (or attributes) for each dimension 
(e.g., eyes might be "wide" or "squinty"). These features were drawn ' 
individually on clear plastic sheets so that by overlaying various sheets, 
a composite face could be constructed. Stimuli were projected on a screen^ 
in front of the subjects using an overhead projector. 

Two task characteristics or stimulus dimensions were chosen for manipu- 
lations: 1) the number of stimulus dimensions which varied (formal difficulty); 
and 2) the extent to which location of features varied from normal appearance 
(perceptual complexity). Three levels of formal difficulty were used: faces 
contained either four, six, or eight facial features which varied from trial 
to trial. There were two possible values for each feature (e.g., hair could 
be either "curly" or 'straight"). Three levels of perceptual complexity 
were utilized. At the first and simplest level, the facial features appeared 
in their normal positions. At the second level the facial features were 
rearranged vertically (e.g., the mouth and eyes were switched), but the 
facial features remained in their normal horizontal orientation. At the 
third and most complex level, the fpatures were moved both horizontally and 
vertically. Examples of each resulting stimulus are presented in Table 1. 

For each problem, a set pf five slides was constructed, each consisting 
of two complete faces placed side by side. Each pair of faces was constructed 
so that both values of each dimension appeared on each slide; ,if the eyes 
were "wide" on the right face, they were "squinty" on the left face. The 
particular association of values on each face varied from sli'de to slide. 
One facial characteristic or dimension (e.g., eyes) and one ^value of that 
dimension (e.g., "squinty") were preselected by the experimenters as the 
solution to each problem. The subj^rt's task was to determine the solution 
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by guessing which of the pair of faces was correct (e.g., contained the \ 
preselected characteristic). After each of the subject's responses, he 
was told which face was correct. Thus, over the course of several trials 
he could find the solution by a process of elimination.* €ach problem 
consisted of five trials {i.e., five pairs o^ faces) with feedback following 
each trial . 

Ten problems each of four, six, and eight dimensions were presented, 
with the first problem in each group always being a sample problem. Of 
the remaining nine problens at each difficulty level, there were three 
prqblems at each perceptual -complexity level. The order of these problems 
was randofnized, with the constraint that no more than two consecutive 
problems were from the same level of perceptual cortplexity. Subjects were 
presented first with the complete set of ten four-dimension problems, then 
all 'qf the six-dimension problems, and finally the eight-dimension problems. 
Problems were presented in this fixed manner in order to avoid the confusion 
which pilot testing indicated would result from the addition and deletion 
of various features from problem to problem. Each subject thus completed 
thirty experimental problems; the first, eleventh, and twenty-first were 
sample problems to introduce the new dimensions. 

Procedure . The experiment was conducted using groups of three to eight 
subjects 3ach day. Subjects first solved a set of four preliminary three- 
dimension problems. While viewing the first pair of faces, subjects were 
told that faces could vary across a number of dimensions, such as the texture 
of the hair (curly or straight), size of the lips (full or thin), or dense- 
ness of the eyebrows (bushy or sparse). They were told that the "correct" 
faces all had one specific facial characteristic in common, and their task 
was to discover that characteristic. Five trials of the facial pairs were 



The arrangement of stimuli in ^ach problem was orthogonal and counterbalanced 
(Levine, 1966). This arrangement provides maximum information to subjects on 
each trial, by switching exactly half the attribute values from one. trial to 
the next. An optimal problem solver had enough information after exactly 
three trials to solve all four-dimension problems. Four trials were the 
minimum necessary to solve the eight-dimension problems; six-dimension problems 
required either three or four trials. 
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then presented. Subjects were told to mark the corresponding left or right 
circle in. their answer booklet -depending upon which face they believed had 
the correct feature. After all subjects had responded for a given trial, 
feedback was given (d.g,, "the left side was correct"). After feedback was 
given for trial 5, subjects were told to write down the one single character- 
istic they thought was the solution to the problem. After each problem was 
completed, the experimenter gave the correct solution. 

On the third preliminary problem, the probe technique was introduced. 
After subjects marked their choice of left-hand or right-hand face and were 
given feedback, they were told to turn to a probe page listing^^all possible 
solutions to the problem and to check all of those facial characteristics 
that they still thought could be the solution to the problem. On the probe 
page for the three-dimension sample problem, the six possible soluticms 
were listed in alphabetical order: bushy eyebrows, curly hair, full rips, 
sparse eyebrows, straight hair, and thin lips, A poster listing the possible 
solutions was placed in a Vocation where all subjects could see it, and 
subjects were urged to refer to it as often as necessary in order to become 
accustomed to the descriptive lables used, Every remaining problem contained 
five probes, one after each feedback. Two additional preliminary problems 
implementing the probe technique were run, with the entire preliminary stage 
of the experiment lasting about 30 minutes. 

Immediately following the preliminary problems, the set of four-dimension 
problems was presented. After a first four-dimension sample problem, subjects 
were shown two slides with features rearranged (one at the second level of 
perceptual complexity and one at the third level), reminding them that the 
features could appear at various levels of disorganization. They then 
proceeded through the remainder of the four-dimension problems and were 
given a ten-minute break. The six-dimensi6n problems were next, and 
once again subjects were shown an initial sa^le problem, followed by two 
slides with features rearranged. Another short break followed, and then, 
the eight-dimension problems were presented, using the same procedure. 

The solutions to each of the thirty experimental ^problems were selected 
randomly, with the following restrictions: all dimensions were used for 
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solution at least once within a given set of 10 problems, no two problems 
in either the six- or eight-dimension set had the same solution, and no 
dimension was used for solution more than three times in the four-dimension 
problems, nor more than twice in the six- or eight-dimension problems. The 
actual order of problems and solutions, as well as thg complete instructions 
,used and a sample f^nswer booklet, are presented in the appendix. After the 
eight-dimension problems, subjects were given a break for lunch. 

Reference Test Battery 

Project stafV re\^iewed definitions of many empirically determined 
abilities', and selected sets of abilities judged as relevant to' the criterion 
task. Tests defining the selected abilities were then combined to form a 
reference battery of 21 tests which was administered to all subjects in the 
afternoon following their participation in the concept-identification task, 
Tne tests represented seven well-established factors in the cognitive, 
perceptual, and memorial domains of performance. To insure adequate factor 
definition, each of the factors was represented by three tests. 

In assembling the battery, considerable use was made of the Kit of 
Reference Tests for Cognitive Factors prepared by French, Ekstrom, and 
Price (1963K While each test taken from the kit had two equivalent forms, 
only one form oY^^ch test was administred due to time limitations. Three 
other tests from aWt of tests developed by Rose (1974) were also included 
in the battery. Factor loadings of these tests from the previous phase of 
the project dealing with performance on an electronic fault-finding task 
(Wheaton, et al . , 1974) were of sufficient magnitude to use them as marker 
tests in the current effort. The Entire test battery was composed of group 
tests of the paper-and-pencil variety. 

Brief descriptions of the reference tests are given below with references 
to further sources of information. The reliability reported for each test 
is shown in Table 2. In cases where this information was unavailable, 
reference was made either to the original test from which/the present version 
c^ime or to a similar test. The order in which the tests were administered 
is shown in the second column. 
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RELIABILITIES OF REFERENCE TESTS 



1 



Induction Factor Order 

Letter Sets Test 8 

Locations Test 6 

Permutations Task 10 

Associative Memory Factor 

Picture-Number Test 3 

Object-Number Test 21 

First and Last Names Test 7 

Flexibility of Closure Factor 

Copying Test 15 
Closure Flexibility 

(Concealed Figures) 20 

Designs Test 14 

Perceptual Speed Factor 

Neisser Search Task 2 

Number Comparison Test 19 

Identical Pictures Test 16 

Syllogistic Reasoning Factor 

Nonsense Syllogisms Test 18 

Inference Test 5 
Gramnatical Reasoning 

(A-B) Task 1 

Spatial Scanning Factor 

Maze Tracing Speed Test 4 

Choosing a Path Test 17 

Map Planning Test 13 

Speed of Closure Factor 

Gestalt Completion Test 11 

. Concealed Words Test 9 

Four Letter Words Test 12 



,64 

•82, 
,83^ 



.76 
,79 
.81 



.88^^ 

.78 
.94 



.80^ 

.79 

.88 



.88^ 
.53 



Source 

Lemke, et a]-. 
Lemke, et al . 
Rose (1974) 



(1967) 
(1967) 



Duncanson (1966) 
Duncanson (1966) 
Duncanson (1966) 



Thurstone (1938) 

Buros (1965) 
Pemberton (1952) 



Rose (1974) 
Duncanson (1966) 
Duncanson (1966) 



Lemke, et al . (1967) 
Guilford, et al. (1952) 



.80^ «^Baddeley (1968) 



.94 Frederiksen (1965) 
.77 Frederiksen (1965) 
.79 Frederiksen (1965) 



.62 Guilford, et al . (1952) 
.80 Guilford, et al. (1952) 
.92 Pemberton (1952) 



lAll reliabilities, unless otherwise indicated, are split-half reliability 

coefficients corrected for full length with the Spearman-Brown formula. 
2Reliability estimated by the tetrachcric correlation of odd and even items. 
3Kuder-Richardson 20 estimate. 
4pearson product moment test-retest reliability. 
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Reference Tests and Ability Factors , The Induction factor has been 
defined as the ability to find general concepts that will fit sets of data. 
It involves the formulation and testing out of hypotheses. The following 
three tests are marker tests for this factor: 

Letter Sets Test— Five sets of four letters each are 
presented. The task is to find the rule which relates four 
of the sets to each other and to mark the one set which does 
not fit the rule. There are 15 items (7 mins.). Score 
is the number correct minus a fraction of the number incorrect 
(French, et al., 1963). 

Locations Test--Each problem consists of five rows of 
small dashes separated into groups of dashes by blank spaces. 
In each of the first four rows, one place in each row is marked 
according to a rule. The task is to discover the rule and to 
mark one of five numbered places in the fifth row accordingly. 
There are 14 problems in all (6 mins.). Score is the number 
correct minus a fraction of the number incorrect (French, et al., . 
1963), 

Permutations Task— In this task, developed by Leskow and 
Smock (1970) and adapted by Rose (1974), the subject is asked 
to write down on separate slips of paper (which he then turns 
face down) as many different license plate numbers containing 
only the digits 1, 2, 3, and 4 as he can think of (3 mins,). 
Scores are: T) the total number of correct new permutations, 
ahd 2) a frequency count of numbers held constant from one 
trial to the next in the first position, and a similar count 
for the second position, given the first was held constant.* 

Associative Memory has been defined as the ability to remember bits of 
unrelated material. The marker tests are: 

Picture-Number Test— The subject studies pictures of coirtnon 
objects, each paired with a two-digit number. Later, when the 
pictures are presented to him in a different order, he is 
required to write in the number associated with them. There 
are 21 items in all (4 mins. for memorizing, 3 mins. for testing). 
Score is the number correct (French, et al., 1963), 

Object-Number Test--The subject studies 20 word-number 
pairs and must recall the appropriate number when the words 
are presented to him in a different order. There are 15 items 
(3 mins. for memorizing, 2 mins. for testing). Score is the 
number correct (French, et al., 1963). 



*The frequency count data, although computed, are not reported in the 
present study. 
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First and Last Names Test--The subject studies 20 full 
names, including firsthand last, and is required to write in 
the appropriate first name when the last names are presented 
in a different arder. There is a total of 15 items (J mins, 
for memorizing, 2 mins. for testing). Score is the number 
correct (French, at al . , 1963). 

The Closure Flexibility factor has been defined as the ability to 
retain a complex idea in spite of distraction. The marker tests are: 

Copying Test--Each item consists of a geometrical figure 
composed of four connecting line segments. The task is to 
copy the figure onto a square matrix of dots. There are 32 
figures (3 mins.). Score is the number correct (French, 
et al., 1963). 

Closure Flexibility Test (Concealed Figures-Form A) — 
Each item consists of a' figure on the left followed by a^ 
row of more complex drawings, some of which contain the 
original figure. The subject marks those drawings which 
contain the figure. Test developed by Thelma G. Thurstone 
a.nd T. E. Jeffrey. There are 40 problems (10 mins,). Score 
is the number correct minus the number incorrect. 

Designs Test--In this test of L. L. Thurstone's (1938), 
300 designs are presented, in 40 of which the Greek capital 
letter "sigma" is embedded. The task is to mark as many as 
possible of the figures containing the "sigma" in a two- - 
minute period. Score is the number correct. 

The Perceptual Speed factor has been described as the ability to 
compare visual configurations and identify two figures as similar or 
identical. The marker tests selected are: 

Neisser Search Task--In this task, developed by Neisser 
(1967) and adapted by Rose (1974), the subject isgiven a 
page containing columns of groups of five letters and is 
asked to search for a particular letter or letters, placing 
a check next to each item (i.e., group) he finds with one of 
the targeted letter(s). There are six trials: in the first 
two trials the subject is given one letter to search for (20 
sees.); in the next two he searches simultaneously for two 
letters (30 sees.); and in the last two, he searches simul- , 
taneously for four letters (30 sees.). The second trial of 
each pair uses a different target (s), but the masking letters 
remain the same. The entire procedure is repeated, using the 
same target(s) but different masking letters. Scores are: 
1) the average time per correct item (in sees.) over all 
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conditions, and 2) the slope of the best fitting V^egression 
line of the time per item (in sees.) by target set size 
function (Rose, 1974),* 

Number Comparison Test--rhe subject examines pairs of 
multi -digit numbers and Indicates whether the two numbers in 
each pair are the same or different. There are 48 pairs of 
items (1 1/2 mins.). Score is the number correct minus the number 
incorrect (French, ht al , , 1963). 

Identical Pictures Test--For each item the subject is 
to check which of five numbered geometrical figures or 
pictures in a row is identical to the reference figure at the 
left end of the row. There are 48 rows or items (1 1/2 mins.). 
Score is the number correct minus a fraction of the number 
incorrect (French, et al . , 1963). 

The Syllogistic Reasoning factor has been described as the abiVty to 
reason from stated premises to their necessary conclusions. The three 
marker tests selected for this factor are: 

Nonsense Syllogisms Test— In this test, suggested by « 
Thurstone's False Premises , the subject is presented with 
formal syllogisms made up of nonsense words so that they 
cannot be solved by reference to past learning. The task is 
to indicate which of the stated conclusions follow logically 
from the premises and which do not. There are 15 items 
(4 mins.). The score is the number correct minus the number 
incorrect (French, et al . , 1963). A constant of 10 was later 
added to each subject's score to eliminate any negative numbers. 

Inference Test--In this test adapted from Guilford, 
the subject's task is to select the one of five conclusions 
that can be drawn from each given statement. There are 
10 items (6 mins.). Score is the number correct minus a 
fraction of the number incorrect (French, et al , , 1963). 

Grammatical Reasoning (A-B) Task--Each item in this 
task, developed by Baddeley (1968) and adapted by Rose (1974), 
consists of a statement follov/ed by a pair of letters 
(either AB or BA). The statements claim to describe the 
order of the two letters (i.<*., to say which precedes or 
* follows the other). The subject's task is to determine 
whether each statement is a true or false description of 
the letter pair which, follows it. . The test is made up of 
two parts, each containing 32 itenis (1 min./part). Score 
for each part is the number correct (Rose, 1974). 



*The slope data, although computed, are not reportc^d in the present study. 
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The Spa.tial Scanning factor has been defined as the ability to visually 
explore a wide or complicated spatial field, A planning ability may also 
be involved/ The marker tests for this factor are: • 

Maze Tracing Speed Test--The task is to find' and mark an 
open path through a moderately complex series of paper mazes. 
There are 24 interconnecting mazes (3 mins,)« Score is the 
number of mazes through which a line has been correctly drawn 
(French, et al. , 1963). 

Choosing a Path Test-'-Each item of this test, adapted from 
AAF Printed Classification Tests (Guilford, et al., Eds.^ 1947), 
consists of a network of lines (as in an electrical-circuit 
diagram) having many intersecting and intermeshed wires with 
several sets of terminals. The task is to trace the lines and 
to determine for which pair of terminals, mafked^S (start) and 
^ F (finish), there is a complete circuit through a circle at 
^ the top. There are 16 items (7 mins.). Score is the number 
of problecns marked correct r'nus a fraction uf the number • 
incorrect (French, et al . , 1963). 

Map Planning Test--In this test, adapted from AAF Printed 
Classification Tests (Guilford, et al., Ec' 1947), the 
subject sees diagrammatic sections representing city maps. 
The sti^eets are blocked at various points by barriers repre- 
sented :by circles. The task is to find the shortest route 
between two given points without crossing any roadblocks. 
There are two maps with ten routes per map (3 mins.). Score 
is the number correct (French, et al., 1963). 

The Speed of Closure factor has been described as the ability to unify 
a complex perceptual field of apparently disparate elements. ^ The marker 
tests are: 

Gestalt Completion TeU--The subject, is required to identify 
and label a number of incomplete pictures under speeded conditions. 
There are 10 items in all (3 mins.). Score is the number correct 
(French, et al. , 1963). 

Concealed Words Test--Words composed of partially obliterated 
letters are presented. The subject is required to write out the 
full word in an adjacent space. There are 25 words (3 mins.). 
Score is the number correct (French, et al., 1963). 

Four Letter Words Test--Twenty-two 46-letter lines of 
capital letters are presented. The task is to circle all the 
4-letter words contained in this array. Score is the number of 
words correctly circled in 2 1/2 minutes (French, 1954).. 
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RESULTS 



The results of three seti, of analyses are presented below. The first 
set deals with the factor structure of the- reference battery. The second 
set is concerned with the impact of experimental , manipulations of the criterion 
task on several measures of performance. The third set describes the re- 
lationships between ability factors and criterion cask performance. 

Reference Battery 

~~~~~--The intercorrelations among reference tests are presented in Table 3. 
Six major factors were extracted from this matrix using a principal com- 
ponents solution. Orthogonal rotation of the factors was performed using 
a varimax criterion. Table 4 presents the rotated factor. loadings; the 
algebraic signs for Factor IV have been reflected for convenience. Factors 
were interpreted for psychological meaningful ness from the projections 
of the reference tests on the rotated axes. 

Factor I is defined primarily in terms of the high loadings exhibited 
by seven of the reference tests. Three of the tests— Copying , Designs, 
and Closure Flexibil ity— have previously been used as marker tests for a 
Flexibility of Closure factor (see Table 2 above and French, et al., 1963). 
The substantial loadings of Maze Tracing, Map Planning, and Choosing a Path 
(included in the battery as marker tests tor a Spatial Scanning factor) 
suggest not only that a broader interpretation of the obtained factor may 
be necessary, but also lend further evidence to Royce's (1973) argument that 
the Flexibility of Closure and Spatial Scanning factors may be components 
of a second-order Visualization factor. It is interesting to note that in 
the previous study (Rose, et al . , 1974), using a somewhat different reference 
battery, the hypothesized Flexibility of Closure and Spatial Scanning factors 
also collapsed into one factor. Jhe high loading of the Identical Pictures 
test on this factor may be dueto the rapid scanning of stimuli and low-level 
processing of the stimuli demanded by the task. Factor I will again be 
tentatively interpreted as a combined Flexibility of Closure/Spatial Scannin g 
factor. 
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TABLE 4 



FACTOR LOADINGS* IN ROTATED FACTOR MATRIX 



Factors 



Reference Tests 

1. Grammatical Reasoning 
Neisser Search 

3 . Picture Number 

4. Maze Tracing Speed 

5. Inference 
6., , Locations 

7 First and Last Names 

8. Letter Sets 

9. Concealed Words 

10. Permutations 

11 . Gestalt Completion 

12. Four-Letter Words 

13. Map Planning 
1^. Designs 

15. Copying 

16. Identical Pictures 

17. Choosing a PaLh 

18. Nonsense S>1 log isms 

19. Number Comparison 

20. Closure Flexibility 

21. Object Number 



T 
I 


1 1 


T T T 
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T\/ 
1 V 


u 

V 


VT 

V i 


h2 

1 i 


16 


20 


15 


72 


17 


24 


69 


22 


05 


74 


03 


-03 


01 


59 


09 


79 


16 


20 


-04 


02 


69 


71 


-03 


17 


07 


19 


18 


60 


15 


14 


-21 


72 


-06 


12 


62 


34 


02 


-14 


06 


14 


66 


59 


-08 


79 


07 


-01 


14 


14 


67 


20 


14 


26 


23 


-02 


68 


64 


16 


10 


22 


05 


82 


-02 


75 


-01 


26 


23 


24 


-07 


50 


43 


30 


-00 


-18 


14 


74 


14 


71 


08 


06 


64 


-13 


32 


17 


58 


55 


06 


02 


35 


01 


39 


58 


71 


20 


09 


28 


09 


05 


65 


78 


15 


15 


08 


15 


22 


73 


75 


-03 


31 


-07 


09 


-06 


67 


46 


-01 


-24 


17 


20 


53 


62 


12 


-19 


02 


72 


15 


06 


59 


35 


17 


63 


05 


-02 


-06 


56 


69 


15 


-02 


27 


26 


37 


78 


24 


83 


-06 


-04 


03 


03 


75 



*Factor loadings reflected and rounded to two places; decimdls omitted. 

Factors are tentatively defined as:\ 

I - Flexibility of Closure/Spatial Scanning 
II - Associative hlemory 
III - Perceptual Speed 
IV - Syllogistic Reasoning 
y - Speed of Closure 
VI - Induction 
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Factor II is readily defined from the high loadings on the Object- 
Number, Picture-Number, and First and Last Names tests as the Ass ociative 
Memory factor. This same factor is defined by French, et al., using the 
same three marker tests, as the ability to remember unrelated bits of 
information. 

Factor III is defined primarily from high loadings of the Neisser 
Search and Number Comparison tests as the P erceptual Speed factor. The 
third marker test hypochesized to load on this factor— Identical Pictures-- 
loaded only marginally (.31), suggesting that the test (with its very 
high loading on Factor I) may be less factorially pure than hitherto 
thought. The Identical Pictures Task also differs from the other tests 
loading on the Perceptual Speed factor in that it involves pictures as 
stimuli, rather than symbols (i.e., numerals or letters). The presence 
of the Four Letter Word test on the obtained factor may be due largely to 
its speed component. The test also broadly resembles those tests loading 
on the Perceptual Speed factor in that it requires the ability to look at 
a visual configuration (i.e., row of letters) and to identify a common 
four-letter pattern, in this sense it is most similar to the Neisser 
Search task, with the primary difference being that in the latter the 
subject searches only for a given letter(s) while in the Four Letter Word 
Test he searches for four letters which form a recognizable word. 

Factor IV is readily defined by the high loadings of the Grammatical 
Reasoning, Inference, and Nonsense Syllogisms tests as the Syllogistic 
Reasoning factor. This factor is defined by French, et al . , as the ability 
to reason from stated premises to their necessary conclusions. 

Factor V is defined primarily by the high loadings of the Concealed 
Words and Gestalt Completion tests as the Speed of Closure factor. Although 
the loading of the Four Letter Words test, also expected to lo=^d on this 
factor, was marginal (.32), this appears to bt the same Speed of Closure 
factor defined by French, et al . It represents the ability to unify a com- 
plex perceptual field. ^ 
/ Factor VI is defined primarily in terms of the high loadings exhibited 

19 



ERIC 



25i 



by the Letter Sets, Locations, and Permutations tests as the Induction 
factor. Frenc*^, et al, , used the first two of these measures as markers 
of an Induction factor and defined it as the ability to find and test out 
hypotheses which will explain sets of data. The relatively high loadings 
of the Choosing a Path and Map Planning tests, which also loaded on the Flexi- 
bility of Closure/Spatial Scanning factor, suggest that neither of the 
tests is a pure measure of a factor and probably involves a complex array 
of abilities in its performance. 

Criterion Task 

Nine raw and derived measures of performance on the concept identifi- 
cation task were available. The first was the proportion of problems solved; 
the other eight were based on the probe data available for every trial of 
every problem. On each probe page of his answer booklet, the checkmarks 
made by a subject were assumed to indicate the hypotheses he held at that 
point about the solution to the problem (Bruner, Goodnow, & Austin, 1956; 
Kornreich, 1968; Wandersman & Wandersman, 1973). Measures based on this 
data reflect additional information about how subjects solved the problems 
under various task conditions. 

All nine of these measures were subjected to analyses of variance; 
all showed a substantial impact of dimensions and perceptual complexity on 
performance. Three dependent measu^^cs wei^e selected as being representative 
of the others, and were used for all subsequent analyses: 

1. Proportion of problems solved under each task condition. 

2. A/B-'-This measure reflected the degree to which subjects tested 
an optimal number of hypotheses on each trial. Since the stimuli 
were constructed to be orthogonal (Levine, 1966), an unambiguously 
optimal strategy could be defined for each problem, and an optimal 
or ideal number of hypotheses on each trial (B) could be derived. 
The number of hypotheses checked on a particular trial (A) was 
divided by B to provide th'? score for that trial . Thus a subject 
who checked three hypotheses on a trial where four were optimal 
received a score of 3/4 "= .75 on that trial. Since showing too 
many hypotheses was not optimal, those few cases in which A 
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exceeded B were corrected by using the following formula- . 
Thus, a subject who showed one too many hypotheses received the 
same score as a subject who showed one too few. This index ranges 
from zero to one, and was available for every trial of every 
problem, as a measure of how optimal the number of hypotheses 

tested by each subject was. 

p 

3, C /AB--This measure yielded a general index of efficiency, available 
at each trial of every problem. The term C was the number of 
hypotheses in a subject's set (A) that were also in the set of 
optimal hypotheses (B). Thus, if a srcrC^'ect checked mustache 
squinty, square, and long (A), and if the optimal set consisted 
of mustache and squinty (B), C for that subject would be two. 
Note then that C/A was the proportion of hypotheses that a subject 
checked that were valid for, or consistent with, an optimal strategy. 

C/B was the proportion of valid or consistent hypotheses that a 

2 

subject checked. C /AB, the product of these two proportions, 
could assume a value between zero and one, and was a measure of 
the strategic efficiency of the actual hypotheses that subjects 
tested on each trial . 

The impact of formal difficulty (number of dimensions) and perceptual 
complexity on criterion task performance was examined in a series of analyses 
of variance on these three dependent variables. In the first such analysis, 
a five-factor analysis of variance was performed using the proportion of 
problems solved as the dependent variable. The five factors were sex (X), 
subjects (S), dimensions (D), perceptual complexity (P), and replications (R)* 



Three factors included in the analyses described in this section (sex, 
replication, and trials) served as control variables to increase the power 
of the .analyses of variance. The primary goal of the analyses was to deter- 
mine whether number of dimensions and levels of perceptual complexity Impacted 
upon group performance, thus indicating the creation of "new" tasks. If "new" 
tasks were created, the experiment would be a reasonable analogue to the Navy 
situation where a new task would require reevaluation of personnel procedures. 
Systematic study of all of the possible effects emerging from these analyses 
lies beyond the scope of this report; therefore, all but an abbreviated 
discussion of the effects of the main variables of interest, as well as the 
-control variables, will be deferred to a later report in this series. 
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The results of this analysis are presented in Table 5. The dimensions 
effect (0) was highly significant, and two linear contrasts were tested to 
determine the nature of the effect, Scheffe test revealed that the four- 
dimension problems (mean proportion solved equaled .864) were solved sig- 
nificantly more often than the (pcoled) six- and eight-dimension problems , 
(means of .685 and .700, respectively, F(l ,252)=35.52, £<.001), while 
there was no significant difference between six- and eight-dimension -problems . 
Although main effect of perceptual complexity (P) was not significant, the 
R X P and D X P X R terms were significant. This suggested that perceptual 
complexity did impact on performance as measured by proportion of problems 
solved, albeit in a •complicated manner, since its influence was modulated 
by level of problem difficulty and replication. 

The results of a six-factor analysis of variance performed on the A/B 
measure are presented in Table 6. Once again the dimensions effect was 
significant. Two linear contrasts were examined to clarify this effect. 
The mean for the four-dimension problems (.856) was significantly different 
from the pooled means (both .800) of the six- and eight-dimension problems 
(Scheffe, F(l ,252)=14.92, £<.001). No significant difference was found 
between the six- and eight-dimension problems. Thus, subjects showed a 
more nearly ideal number of hypotheses when solving four-dimension problems 
than when solving six- or eight-dimension problems. 

No significant main effect was found for perceptual complexity using 
the A/B dependent variable, but a significant D x P interaction was obtained. 
This interaction is depicted in Figure 1. A set ot simple main effects 
tests (Kirk, 1968) was performed on the interaction to examine the effect 
of each independent variable at every level of the other independent variable. 
The effect of , dimensions was found to be significant at every level of 
perceptual complexity [F(2,381 )=3.292, p<.05; F(2,381 ) = 16.867, £<.001; 
F(2,381 )=28.207, P<.001, for levels one, two, and three of perceptual 
complexity respectively]. The effect of perceptual complexity was found to 
be significant only for four- and eight-dimension problems (F(2,381 )=10.458, 
p<.001 and £(2,381 )=3. 972, p<.025, respectively). Tukey's HSD statistic 
(Kirk, 1968) was used to test pairwise v/ithin each of the significant 
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TABLE 5 



ANALYSIS OF VARIANCE OF CRITERION DATA 
(Proportion Solved) 



Source 



Between Subjects: 
X (Sex) 
S/X 

Within Subjects: 
D (Dimensions) 
DX 

OS/X 



P (Perceptual Complexity) 

PX 

PS/X 

DP 

DPX 

DPS/X 

R (RepUcations) 
RX 

RS/X 

OR 

DRX 

DRS/X 

PR 

PRX 

PRS/X 

DPR 

DPRX 

DPRS/X 



df 



1 

126 



MS 



4 .000 

1.008 



000 



2 


11 .444 


53 . 909*** 


2 


.125 


.588 


252 


.212 




2 


.217 


1 .435 


2 


.049 


.323 


252 


.151 




4 


. C.i 1 


1 81 0 

1 • O 1 V/ 


4 


.214 


1.674 


504 


.128 




2 


.964 


6.956*** 


2 


.580 


4.187* 


252 


.139 




4 


.211 


1 .380 


4 


.374 


2.447* 


504 


.153 




4 


1.056 


6.845*** 


4 


.121 


.781 


504 


.154 




8 


.545 


4.056*** 


8 


.073 


.545 


1008 


.134 





* p < .05 
** p < .01 
*** p < . 001 
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TABLE 6 



ANALYSIS OF VARIANCE OF CRITERION DATA 
(A/B) 



Source 


df 


MS 


F 


Between Subjects: 
X (Sex) 
S/X 


1 

126 


1 .032 
3.270 


.31b 


Within Subjects: 
D (Dimensions) 
DX 

DS/X 


2 

2 

252 


6.475 
1 ,792 
.269 


24.1 07^*** 
6.672** 

r 


P (Perceptual Complexity) 

PX ■ 

PS/X 


2 
2 

252 


.095 
.011 
.092 


1.030 
.1-21 


DP 

DPX 

DPS/X 


4' 
4 

504 


.662 
.265 
.093 


7.087*** 
2.833* 


R (Replications) 
RX 

RS/X 


2 
2 

252 


.328 
.074 
.118 


2.775 
.629 


OR 

DRX 
DRS/X 


4 

4 

504 


.486 
,196 
.101 


'4.810*** 
1.944 


PR 

PRX 

PRS/X 


4 
4 

504 


1 .916 
.117 
.101 


19.005*** 
1.164 


DPR 

DPRX 

DPRS/X 


8 

, 8 
1008 


.738 
.100 
.095 


7.802*** 
1.053 


T (Trials) 
TX 

TS/X 


4 
4 
504 


14.093 
3.443 
.528 


26.689*** 
6.520*** 


DT 

DTX 

DTS/X 


8 
8 

1008 


,781 
.205 
.099 


7.889*** 
2.070* 



* p < .05 
** p < .01 

*** p < .001 24 



TABLE 6 (Cont..) 
ANALYSIS OF VARIANCE OF CRITERION DATA 



,. (A/B) 


Source 


df 


MS 


F 


PI 

PTX 
PTS/X 


8 
8 

1008 


. 118 
.042 
.053 


.788. 


DPT 

DPTX 

DPTS/X 


16 
16 
2016 


.156 
.027 
. 055 


.499 


RT 

«TX 

RTS/X 


8 
8 

1008 


.224 
.105 
.055 


4 . 063*** 
1.911 


DRT 

DRTX 

ORTS/X 


16 
16 
2016 


.104 
.030 
.058 


1.795* 
.508 


PRT 

PRTX 

PRTS/X 


16 
16 
2016 


.252 
.048 
, .053 


4.724*** 
.906 


DPRT 

DPRTX 

DPRTS/X 


32 
32 
4032 


.180 
.043 
.055 


3.283*** 
.775 
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simple main effects. At all levels of perceptual complexity, four- and six- 
dimension problems were significantly different (£ (381 ) = 3.62, £<.05; 
a (381) = 7.47, £<.01 ; g. (381 ) = 7.80, £<.01 , respectively), while six- 
and eight-dimension differences were never significant. Four-dimension 
problems were significantly different from eight-dimension problems only 
at the second and third levels of perceptual complexity (£ (381) = 6.70, 
£< .01; (381) = 10.17, p^<.01). Within the four-dimension problems, 
level one complexity was significantly different from level two (£ (381) = 
4.18, £<.01),' but levels two and three were not different; at six dimensions 
none of the levels of perceptual complexity was significantly different; 
at eight dimensions, the only significant difference was between levels 
one and three (£ (381) = 4,06, £<.05). In summary, a more optima-1 number 
of hypotheses was shown ia four-dimension problems than in six or eight, 
and, while subjects approached the ideal number of hypotheses as perceptual 
complexity increased on four-dimension problems, they behaved less optimally 
as complexity increased on the eight-dimension problems. As dimensions 
(difficulty) increased, optimal performance changed from an increasing 
to a decreasing function of perceptual complexity. 

2 

A six-factor analysis of variance was also performed on the C /AB depen- 
dent variable, and the results are presented in Table 7. On this measure, 
the main effects of both dimensions and perceptual complexity were signifi- 
cant. Scheffe tests of linear contrasts showed that subjects were more 
efficient (F (1 ,252) = 42.58, £<.001) on four-dimension problems (mean of 
.773) than on six- or eight-dimension problems (means of .643 and .630, 
respectively), but that there was no significant difference in performance 
on the six- and eight-dimension problems. Additional Scheffe comparisons 
on the perceptual -complexity effect revealed that level one (mean of .659) 
differed significantly (F (1 ,252) = 5.17, £<.025) from levels two and three 
(means of .699 and .687, respectively), while there was no significant 
difference between levels two and three. Thus, it seems that while efficiency 
decreases as the number of dimensions (difficulty) increases, efficiency 
increases as perceptual complexity increases. 

These relationships are further clarified in Figure 2, which shows the 
significant interaction of perceptual complexity with replications. If the 
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TABLE 7 



ANALYSIS OF VARIANCE OF CRITERION DATA 
(C2/AB)' 



Source 


df 


MS 


F 


Between Subjects: 
X (Sex) 
S/X 


1 

126 


3.632 
-5.845 


.621 


Within Subjects: 
D (Dimensions) 
DX 

DS/X 


2 
2 

252 


36.764 
1.409 
.560 


6 5. .6 2 5*** 
2.515 


P (Perceptual Complexity) 
PX 

PS/X 


2 
2 

252 


2.407 
. 031 
.286 


8.416*** 
.108 


DP 

DPX 
DPS/X 


4 
4 

504 


.175 
.583 
.254 


.688 
2.292 


R (Replications) 
RX 

RS/X 


2 
2 

252 


1.375 
.623 
.297 


4.626* 
2.094 


DR 

DRX 
DRS/X 


4 
4 

504 


1 .221 
1 .007 
.309 


3.948** 
3.257* 


PR 

PRX 

PRS/X 


4 

4 

504 


5.147 
.286 
.260 - 


19.778*** 
1.099 


DPR 
DPRX 
. DPRS/X 


8 
8 

1008 


r 3.086 
.127 
.270 


11 .433*** 
.470 


T (Trials) 
TX 

TS/X 


4 
4 

504 


16.442 
1 .926 
.239 . 


68.829*** 
8.061*** 


DT 

DTX 

DTS/X 


8 
8 

1008 


1.054 
.077 
.077 


13.684*** 
T.003 



* p < .05 
** p < .01 
*** p < .001 
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TABLE 7 (Cont. ) 



ANALYSIS OF VARIANCE OF CRITERION DATA 
(C2/AB) 




df 


MS 


F 


PT 


8 


.207 


3.827*** 


r 1 A 


8 


.049 


.907 


PTS/X 


1008 


.054 




DPT 


16 


. 222 


4.11 0*** 


nPTY 


16 


i088 


1 .622 


DPTS/X 


2016 ' 


.054 




RT 


8 


.244 


3.900*** 


r\ 1 A 


8 


.106 


1 .693 


RTS/X 


1008 


.063 




DRT 


16 


.373 


6.357*** 


DRTX 


16 


.113 


1.925* ' 


DRTS/X 


2016 


.059 




PRT 


16 


.324 


5.893*** 


PRTX 


16 


.044 


.795 


PRTS/X 


2016 


.055 




DPRT 


32 


.306 


5.430*** 


DPRTX 


32 


.035 


.625 


DPRTS/X 


4032 


.056 
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first replication of the third If^vel of perceptual complexity is ignored, 
efficiency clearly increa'^es as a function of perceptual complexity This 
is borne out by several post-hoc tests perforpied on ihe interaction. For 
example, when level-one perceptual complexity is collapsed across the second 
and third replications, and compared with level three similarly collapsed, 
the level-one problems are solved significantly less efficiently (Scheffe, 
F (1.504) = 9.63, £ .005), When levels two and three are compared, 
collap 'jd simila»^ly across the second and third replications, the difference 
approaches significance (Scheffe, F (1 ,504) = 3.26, £<J0). 

The analyses performed on the -three selected dependent variables, pro- 
portion of problems solved, number of hypotheses (A/B), and efficiency of 
selected hypotheses (C /AB), reveal many strong effects on performance due 
to" the two task variables. In general, increasing the number of c'imensions 
from four to six or eight impairs performance across all three measures. 
Increasing the perceptual complexity results in different effects for the 
two probe measures: added complexity results in an increase in processing 
efficiency, while the effect on numbers of hypotheses varies as a function 
of problem difficulty. " Finally, while many first- and higher-order interactions 
of Indications with these task variables were significant, suggesting a more 
complicated picture, these effects were not of immediate concern in the 
present study. 

P rojection of Criterion Data on Reference Factors 

The purpose in conducting this final set of analyses was to relate 
variation in the two criterion task characteristics to the pattern(s) of 
abilities contributing to performance. While an analysis of the relation- 
ship between abilities and performance in each cell of the design was 
possible, only th:. relationships of abilities to changes in the dimension 
and perceptual -complexity variables were explored by examining the main 
effects and their interaction* 

A Str ff regression procirdure (Stoloff, 1 973) was performed to obtain 
the estimated loadings of the difficulty and perceptual -complexity effects 
as well as the D x P interaction on the factor structure underlying the 
refere-ice battery. The coefficients estimated by the Stoloff proredurc 
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for the main effects on each of the three dependent variables are presented 
in Table 8. The communal ities (h ) are generally high, and indicate that 
from 24:o to 45:^ of the variance in criterion task performance can be accounted 
for by individual differences on tne six, factor-defined abilities.^ Since 
coefficients For Factors III and V (Perceptual Speed and Speed of Closure) 
are uniformly low, most of the variance in performance is accounted for 
by the Flexibility of Closure/Spatial Scanning (I), Associative Memory (II), 
Syllogistic Reasoning (IV), and Induction (VI) factors. 

To assist in determining the patterns of these loadlng^s, they are 
plotted in Figures 3, 4, and 5, for proportion solved, A/B, and C^/AB, 
respectively (loadings on Factors III and V are omitted). From these figures 
several conclusions may be drawn. The loadings on Factor I (Flexibility 
of Closure/Spatial Scanning) are at best n^^dprate for all levels of formal 
difficulty, increasing from four- to six-o nsion problems and decreasing 
from six- to eight-dimension problems. Although still modest in size, the 
loadings on Factor I increase across levels of perceptual complexity for 
proportion solved, as well as for the measure of efficiency in task per- 
formance (C /AB). Loadings for the A/B measure across perceptual complexity 
on this factor are ..loderate and remain relatively constant. 

The loadings for Factor II (Associative Memory) are fairly high for 
all three dependent measures, generally increasing as difficulty increases 
(especially for A/B), while remaining quite constant over levels of perceptual 
complexity. The loadings on Factor IV (Syllogistic Reasoning) are reasonably 
high for the two dependent variables based on the probe data (A/B and C /AB), 
and modest for proportion solved., The loadings on this factor decrease 
from level one to level two of perceptual complexity, and increa<^c from 
level two to level three. The opposite pattern is found (an increase and 
then a der.'^ase) as formal difficulty increases. Finally, loadings on 
Factor VI (Induction) are low and remain fairly constant as perceptual 
complexity increases; Factor VI loadings tend to rem.ain fairly constant 
for the two probe data measures as formal difficulty increases, while 
increasing for the proportion-solved measure, 
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TABLE 8 



ESTIKATED LOADINGS* OF CRITERION VARIABLE MARGINALS 
ON REFERENCE FACfOR STRUCTURE 



Factors 



Criterion Variables 


I 


II 


III 


IV 


V 


VI 


.2 
h 


Proportion Solved; 


















Perceptual Com- 
















.27 


plexity 


1 




. 00 


1 R 


29 


1 1 
■ 1 1 


.17 


2 


.27 


.34 


.00 


.20 


.12 


!25 


.30 




3 


.30 


.33 


-.01 


.31 


.06 


.20 


.33 


Dimensions 


4D 




pi 


13 


21 


.17 


.16 


.24 




6D 


.32 


.37 


.00 


.27 


.01 


.16 


.34 




8D 


.15 


.32 


.02 


.27 


.11 


.26 


.28 


A/B :' 


















Perceptual Com- 












.10 


.25 


.37 


plexity 


1 


25 


28 


.15 


.36 


2 


.31 


.26 


.14 


.35 


.08 


.28 


.39 




3 


.26 


.28 


.15 


.36 


.09 


.27 


.38 


Dimensions 


4D 


.24 


.15 


.23 


.36 


.02 


.28 


.34 




6D 


'.29 


.27 


.09 


.38 


.10 


.25 


.38 




8D 


.26 


.34 


.11 


.28 


.13 


.24 


.35 


C^/AB :; 


















Perceptual Com- 












.14 


.24 


.41 


plexity 


1 


.17 


.37 


.15 


.38 


2 


.27 


.39 


.07 


.32 


.09 


.30 


.43 




3 


.28 


.35 


.04 


.35 


.10 


.26 


.40 


Dimensions 


4D 


.22 


.29 


.15 


.35 


.09 


.27 


.36 




6D 


.30 


.40 


.04 


.35 


.10 


.24 


.45 




8D 


.18 


.36 


.07 


.31 


.12 


.26 


.35 



* Signs have been reflected to relate superior performance to superior 
abil ity. 



** Factors are identified as:; I - Flexibility of Closure/Spatial Scanning; 
II - Associative Memory; III - Per-eptual Speed; IV - Syllogistic Rea- 
soning; V - Speed of Closure; VI - Induction, 
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Since the analyses of variance revealed interactions between the two 
task characteristics, Stoloff loadings were obtained to measure the rela- 
tionship between performance in each of the nine D x P cells and the 
factor structure unc lying the reference battery. The communal ities 
for these regressions are presented in Table 9, The loadings are pre- 
sented for the proportion-solved measure in Figures 6a through 6f (one for 

2 

each factor), for A/B in Figures 7a through 7f, and for C /AB in Figures 
8a through 8f • 

The communal ities in Tables 8 and 9 indicate that while the proportion 
of variance accounted for is somewhat less for the interaction than for 
the main-effect loadings, they are still reasonably high, ranging from 
15% to 41%. In addition. Figures 6, 7, and 8 indicate that the cell loadings 
follow the marginal loadings for the most part, but some deviations do 
occur. 

Cell loadings on the dependent variable proportion solved (Figure 6) 
may be compared with the marginal loadings shown in Figure 3, The cell 
loadings for Factor I (Figure 6a) follow the marginals quite closely, rising 
and then falling as a function of difficulty, and rising as perceptual 
complexity increases. The one exception is the six-dimension, level-three 
loading, which is lower than might be expected from the marginal pattern. 
The cell loadings for Factor II (Figure 6b) also follow the difficulty 
marginal loading pattern, with the exception of the six-dimension, level- 
three cell. They do, however, seem more variable across levels of perceptual 
complexity. The loadings for Factor III (Figure 6c) are still low, but 
an interesting pattern may be present. The cell loadings for Factor IV 
(Figure 6d) are considerably more variable than is suggested by the marginal 
loadings, .indicating that the D x P interaction strongly influences the 
degree of involvement of this factor. The cell loadings for Factor V 
(Figure 6e) are low, as were the marginal loadings, but there is a hint that 
this factor may be involved in very easy problems, or hard and complex 
problems, but not otherwise. Factor VI cell loadings (Figure 6f) suggest 
a moderate rise in loading as difficulty increases for the intermediate 
level of perceptual complexity. 
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TABLE 9 

COMMUNALITIES (h^) OF ESTIMATED REGRESSION EQUATIONS FOR D x P INTERACTION MATRIX 



Criterion 
Variable 


Level of Perceptual 
Complexity 


Number of Dimensions (Formal Difficulty) 


Four 


Six 


Eight 


Proportion 1 
Solved 

2 

3 


.15 .20 .20 
.20 .25 .26 
.24 .24 .17 


A/B 1 

. / 2 
1 

/ ^ ■ 


.34 .33 .25 
.33 .35 .28 
.22 .33 .41 


C2/AB / 1 

/ ' • 

' 3 


.30 .35 .28 
.36 .37 .34 

.29 .33 . .32 
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The cell loadings for the dependent measure A/B (Figure 7) follow 
the marginal loading patterns almost exactly. This might seem somewhat 
surprising, as it was this measure of performance which was spnsitive to 
the D X P interaction (cf. Figure 1), However, it is important to note 
that the analysis of variance is sensitive to group performance, while 
the Stoloff regressions describe the relationship between individual abilities 
and performance. 

As shown in Figure 8, the cell loadings for the third performance 
measure, C^/AB, are almost identical in pattern to those for the first per- 
formance measure, proportion solved, and all of the comments applied to 
Figure 6 apply to this one as well. One exception is Factor VI, which has 
moderate and generally constant loadings for all cells. 

While the proportion of variance accounted for in this study is relatively 
substantial, the patterns which emerge in the loadings are not easily summarized. 
Perceptual Speed and Speed of Closure (Factors III and V) show very little 
relationship to performance on the criterion task, while the other factors 
all show moderate to fairly strong involvement which varies as a complex 
function of the task manipulations. 
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DISCUSSION 



Formal difficulty strongly and systematically influenced performance.- 
on the criterion task. ^In general, performance on each of the three dependent 
measures deteriorated as the number of dimensions v/as increased from four 
to six. The expected impact of eight dimensions was apparently offset by 
practice; subjects had almost twice as much experience with the task when 
they began the eight-dimension problems as when they began the six-dimension 
problems. The perceptual complexity task characteristic also impacted on 
criterion performance, but its influence was more subtle. On the number-of- 
hypotheses measure (A/B) subjects did better with increased complexity 
on easy problems, and worse as complexly increased on difficult problems. 
An improvement in performance was found for the efficiency measure (C /AB) 
as perceptual complexity increased regardless of problem difficulty. No 
simple effect of perceptual complexity on the proportion-solved measure was 
found. 

While a general decrement in perfornidnce as number of dimensions increases 
is a consistent finding in concept identification research (cf. Bourne, et 
al., 1971), the impact of perceptual complexity has not been studied 
extensivp^ly. A hypothesis to explain the present results can be developed 
based on theoretical work which has been done in concept identification 
tasks (Bruner, et al , , 1956; Levine, 1969). Subjects sample a subset of 
the possible solutions, and test this subset using information from 
successive feedback trials. This subset may include one or more hypothesized 
solutions, and is known as the focus set. The optimal focus set size (all 
of the attributes on the first trial, half on the second, one-fourth on 
the third, and so on) serves as a baseline for the A/B performance measure 
used in the current study. This measure reflects the actual size of a 
subject*s focus set (A) relative to the optimal set size (B). According 
to Levine (1970), subjects test the hypotheses in their focus set, and thus 
narrow it down, in the following fashion. On each trial, the subject 
chooses one side of the stimulus, and codes those facial attributes which 
are in his focus set and on the chosen side. If his choice response is 
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correct, he retains the coded attributes in his focus set, and eliminates 

the uncoded attribute:>; if his choice response is incorrect, he eliminates 

the coded attributes and retains the others as his updated focus set. The 

2 

performance measure C /AB indicates the number of attributes that are 
consistent with all preceding feedback information and which the subject has 
correctly retained (C), relative to the total number which he has retained 
(A) and the number which he should optimally have retained (B). 

2 

Thus, A/B measures the size of a subject's tocus set, while C /AB 
measures how well he uses feedback information to correctly narrow down 
his set to the solution to the problem. The number of problems solved 
depends on the size of the focus set and the proper elimination of incorrect 
attributes from the focus set; thus the proportion-solved measure is partially 
dependent on ti e other two mea.ures. When a subject codes the intersection 
of his focus set and the chosen face, he must find the attributes of interest 
and remember theni as distinct from his total focus set so that he may properly 
apply the feedback information, In the easiest and least complex problems, 
it seems reasonable to suppose that finding the attributes in the normal 
face is quite simple, and that they are remembered (or rehearsed) in 
accordance with the order in the face itself, e.g., top-to-bottom. While 
the normal face presentation does not explicitly require that the list of 
attributes be remembered in top-to-bottom order, the subject may feel 
compelled to do so since it seems more 'natural.' As the perceptual com- 
plexity increases, this natural ordering is lost. It is generally held 
(e,g,, Postman, 1972) that when no list ordering is imposed by the experi- 
menter, the subject organizes the list himself so that it is more easily 
remembered. According to Postman, "The organization imposed by the learner 
on a set of items depends on his perception of the structure of the list.,. 
For any list of items presented to' a particu lar subject there is an optimal 
organization that will maximize recall" (p. 5, emphasis added). Thus, when 
the naturally ordered face is disrupted, subjects are free to (or forced to) 
actively organize the list themselves. Empirical findings (Tulving & Donaldson, 
1972) suggest that this active involvement in problems with high perceptual 
complexity produces better recall than does the passive use of the top-to- 
bottom orrjanization provided in problems at levpl one of perceptual complexity. 



Obviously, the more attributes remembered, the more likely that feedback 
would be used correctly to narrow down the focus set. This is reflected 
in improved performance on the C"//^B measure. 

In order to account for the results on the A/B measure, ore additional 
assumption is necessary; the subject's approach to the problem (i,e,, the 
number of hypotheses he chooses to test simultaneously in his focus set) is 
dependent on the total cognitive load imposed by the task. When the number 
of total hypotheses is small, as in the four-dimension problems, the added 
load of finding the desired attributes and constructing a list organization 
is easily borne, and recall of the attributes is made easier due to the 
subject-generated list organization. The net cognitive load is decreased, 
thus allowing the subject to attempt to handle a larger focus set. When 
more totil possible solutions are involved, as in the eight-dimension prob- 
lems, the burden of constructing a list organization plus finding the attri- 
butes causes the subject to reduce his net load by reducing the size of his 
focus set. This would account for the observed changes in performance on 
the A/B measure. 

There is some support fcr these hypotheses in the Stoloff relationships 
between abilities and performance as task characteristics are varied. The 
ability to scan and process complex visual fields (Factor I) becomes more 
predictive of overall performance (proportion solved) as perceptual com- 
plexity increases. However, it does not become more predictive of the focus- 
set measure (A/B) as complexity increases, either for the complexity mar- 
ginal, or for complexity at individual levels of difficulty. This suggests 
that the added load of memory organization construction is more important 
in accounting for the D x P interaction for the A/B measure. Factor I does 
increase its relationship with the C /AB measure as complexity is increased, 
suggesting that subjects have difficulty processing attributes as they be- 
come more difficult to find, but that they do not change their approach when 
this occurs. 

In accordance with these hypotheses, the Associative Memory factor (II) 
does become more predictive of the size of the subject's focus set as well 
as of his p»^ocessing efficiency as formal difficulty increases. Thus, it 
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becomes increasingly involved in both the approach the subject adopts, and 
how well he executes that approach. The non-monotonic relation of Factor II 
with proportion of problems solved over levels of formal difficulty is not 
understood at present. 

While the ability data may be helpful in understanding the effects 
of the task-characteristic changes on performance, the goal of the study is 
to examine the effects of task-characteristic changes on the ability require- 
ments. Thus, as task characteristics were varied, systematic changes in 
performance were noted, and the ability patterns predicting performance also 
changed. Are these changes in ability patterns systematic, and does it 
seem (at least ultimately) that they would be predictable from knowledge 
of the task characteristic manipulations? 

In order to examine the results of this study from this point of view, 
it will be useful to review the possible relationships which might be found 
between changes in task characteristics and changes in degree of involvement 
of one or more abilities. One such relationship occurs when, as a result 
of a task characteristic change, a new task is created which requires abili- 
ties entirely different from those required by the original task. In this 
case one or more abilities which predicted performance in the old task would 
have essentially zero loadings in the new one, while others which were unin- 
vblved in performance on the original task would predict performance in the 
new version. Examples from the current study might include Associative 
Memory, which has a very low loading on A/B performance for four-dimension 
problems, but which has moderate loadings for six- and "eight-dimension 

problems. As another example, the loading for Flexibility of Closure/ 

2 

Spatial Scanning on the proportion-solved and C /AB measures is very 
low for problems at the lowest level of perceptual complexity, but rises 
for more complex problems. 

Another pattern of relationship between task characteristics and abilities 
occurs when, as a characteristic is manipulated, one or more abilities gradually 
rise or fall in importance. This pattern is essentially indistinguishable 
from the first, since a true zero correlation (indication of non-involvement) 
is empirically most unlikely. However, the distinction is theoretically 
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important, since the first pattern implies that a threshold has heen crossed, 
und the new task is truly different from the old, while, a more gradual change 
implies a continuity between versions of tasks. As long as one uses a hetero- 
geneous population of subjects and several levels of edch task characteristic, 
an abrupt change in loadings is empirically unlikely, since aJJ^ subjects 
would simultaneously have to reach the implied threshold, with corresponding 
identical changes in task characteristics^ 

A third pattern of relationship between characteristics and abilities 
occurs when, as the task is manipulated, one or more abilities change 
non-monotonically. One need only glance through Figures 3 through 8 of the 
current study to find many such instances of this kind of characteristic- 
ability pattern. For example, the loadings of Factor I on all three per- 
formance ^^easures rise and then fall as difficulty increases. Such 
patterns, of course, might arise were the task characteristics in fact 
not correctly ordered; in our case this might be reasonable for the per- 
ceptual manipulation, but probably not for difficulty. 

Wheaton, et al . (1973) invoke the concepts of intrinsic and extrinsic 
task variations in an attempt to explain different ability-task character- 
istic loading patterns. Given a set of abilities which are involved 
in a task, an extrinsic variation produces monotonic variations in the set 
of abilities; whether the loadings increase or decrease, they change in a 
similar fashion for all abilities in the set. Extrinsic variations may 
also produce more complex patterns, when the variations are improperly 
ordered, or when the range of variation is very great. "Under the siiiplest 
conditions, all subjects might perform relatively well, despite different 
levels of ability. Under the most difficult conditions, performance would 
be generally poor, no matter how much ability a given subject possessed • 
Between these extremes the advantage would be with those subjects possessing 
the greatest amount of the relevant abilities" (p.^ 35). 

While extrinsic variations may produce similar changes in loadings for 
an entire set of abilities, intrinsic variations produce different changes 
for different abilities in the set. Intrinsic variations are hypothesized 
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to produce this kind of change because the task, from the subjects* point 
of view, has changed. Subjects may then apply a new approach to this new 
and different task, which calls a new set of abilities into play. In the 
present study, therefore, both formal difficulty and perceptual complexity 
seem to represent intrinsic task variations, since the set of abilities do 
not covary consistently. Rose, et aU (1974) reported similar findings 
for a fault-finding task; as formal difficulty and perceptual complexity 
were varied, no consistent covariation of ability loadings on performance 
were observed. Wheaton, et al . found only one ability related to the per- 
formance of the tasks they examined, and so were precluded from examining 
covariation in a set of abilities. 

Rose, e^ al. made an additional finding which bears on tnis discussion. 
They found, as did the present study, that different measures of performance 
on the criterion task showed different effects of varying task characteris- 
tics, and led to different characteristic-ability interactions. Any descrip- 
tion of changes in ability loadings as a task characteristic varied depended 
upon the measure of performance which was used to estimate the loading. 
They suggested that the different measures of performance represent different 
aspects of a subject's approach to the problem. As a task characteristic 
varied, one aspect of the subject's approach might change, leading to niw 
ability requirements. Those same task variations need not have changed 
other aspects of that subject's approach, leaving 6ther abi'^'ity require- 
ments unchanged. It is even possible to iinagine a case in which two 
independent aspects of a subject's approach might change in response to a 
particular task variation so as to create completely contradictory loading 
changes on the same set of abilities' for two different measures of performance. 
The picture becomes even more complicated when one allows for two task 
variables which interact, or when the changes in a particular subject's 
approach depend on the actual amounts of various abilities which he possesses. 

Rose, et al. suggest that the proper mediator of task-ability ^elation- 
ships is the subject's approach. From a cognitive point of view, the way 
he views the task determines the way in which he will attempt to deal with 
it, and this in turn will detennine the abilities which are called into 
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play. If a task variation does not change the nature of the task from the 
subject's point of view, no change in the set of ability requirements will 
be observed, although this set may become more or less predictive; if the 
task is seen as a new one, the set of abilities required by the approach 
adopted for this new task may be entirely different. For example, one 
would expect abilities related to speed of processing to be related to 
performance in a task where the subject's approach included an emphasis 
on speed. In the present study, no need for speed was imposed, and no 
relation to Perceptual Speed or Speed of Closure factors was found. Suppose 
that, in the present study, another version of the task were used which 
required that subjects work as quickly as possible. In this case one 
might expect Factors III and V to predict some aspect of performance on 
this version of the criterion task. 

The admittedly speculative analysis presented at the beginning of this 
section is another example of this kind of analysis. That explanation supposes 
that certain combinations of task variations cause subjects (on the average) 
to change their approach. As aspects of their approach change, changes 
appear in the loadings of relevant abilities-on the dependent measures which, 
in part, reflect those hypothesized changes in approach. 

Actually, a much more efficient way of ^amining the validity of this 
kind of analysis is available for the data collected in the current study. 
A great deal of theoretical and empirical work exists regarding the behavior 
of subjects in solving concept identification tasks (e.g., Bruner, et al . , 
1956; Gholson, Levine, & Phillips, 1972; Gholson, Phillips, & Levine, 1^/3; 
Levine, 1969, 1970; Restle, 1962; Wandersman & Wandersman^, 1973; Wickens & 
Millward, 1971). Researchers have described various ways in which subjects 
code the members of their focus sets, process feedback information, and 
remember information from preceding trials. Perhaps the most extensive 
work has been done in describing what happens when subjects select focus 
sets of various sizes, how sets are narrowed down, and the kinds of 
strategies used to locate the solution. 

From the data available in the present study, it is possible to identify 
when subjects might have used one or another of these strategies, and how 
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these might have changed for individual subjects as the task variations were 
introduced. Based on this data it should be possible to ascertain more 
precisely what happens when a task v^ariation causes a subject to change 
his approach. In addition, it may be possible to relate both theoret^ lly 
and. empirically the ability requirements of several of the approaches adopted 
by subjects. 

The following kinds of analyses are. planned in the next phase of this 
project to attempt to validate the change-of-task, change-of-approach, 
change-of^ability model. One technique will be to stratify subjects on the 
basis of their approach to various versions of the task to determine if 
changes in ability loadings on performance coincide with changes in approach. 
Subjects will also be stratified on the basis of certain ability patterns, 
to examine the way that abilities which subjects possess might influence 
their choice of approach. Subjects may also be stratified based on their 
performance on one or more of the dependent variables; these samples 
would then be examined ^or approaches used and abilities possessed. 

While the particular kinds of approaches iused by subjects in concept 
identification tasks are of limited general izability to tasks in the Navy, 
two very important outcomes are expected. If the importance of change 
of approach to change in ability requirements can be validated, it may 
become possible in the short term to predict whether a new version of an 
old Navy task will require revision of personnel decision-making criteria. 
Based on empirical and theoretical analysis of the tasks involved, it may 
be possible to determine if, for example, job incumbents would view a 
variation in the old task to have changed its nature, and thus change their 
approach. If incumbents were not expected to change their approach, no 
new selection instruments or training courses would be required (although 
performance criteria in selection or training might require change). 

If a change in approach is predicted to oc^ur, the problem becomes much 
more difficult. While personnel planners would know that their current 
procedures were inadequate, determining new requirements. would depend on 
ascertaining which new approach would be employed, and analyzing that 
approach for ability requirements. This process would seem to require a great 
deal of additional research before it could be realized. 
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This appendix contains three items to further document the procedure 
used in administering the concent identification criterion task. First, 
a detailed description of the procedure, including complete instructions, 
is presented. Next, a table is provided which indicates the ordBr-tn ' wh i c h 
subjects received problems, and the solutions tJi t-ho^e problems. Finally, 
several pages from a subject answer booklet are included. Subjects solved 
up to six problems per booklet, and responses for each of these were 
recorded in a different column. Thus, on each trial the subject turned 
to a new page; for each new problem he returned to the front of the booklet 
and used the next column. Included are a sample choice response page, 
a sample probe page, and a sample trial-five probe page, which includes space 
for writing the solution.. The samples provided are for eight-dimension 
problems. Others would be identical, except that fewer attributes would be 
listed on the probe pages. 
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PROClDURE':; EXPERIMENTAL CRITERION TASK 



^^^•^ Subjects were held in a wditing area until the whole group was assembled. 

They then entered the laboratory where the experimenters introduced them- 
selves. The following instructions were read aloud. Questions were answered 
by repeating or rephrasing appropriate sections froni the instructions. 

Instr uctions 

You are about to participate in one phase of a continuing research 
effort designed to help us understand the relationships between various tasks 
and the abilities necessary to perform these tasks. As you know, the 
experifnent in which you are taking part today is concerned with problem 
solving. Your morning session will consist of a scries of these problem 
solving tasks with several rest breaks interspersed. Then you will have a 
lunch break after which you will be taking 21 paper-and-penci 1 tests covering 
many aspects of human ability. These tests were selected because they are 
either known or hypothesized to be related to performance on the problem 
solving task.^ 

We anticipate finishing by about 5:00 p.m. You will be paid $20.00 
in cash at that time for your participation. Are there any questions? We 
are now ready to begin the problem solving task. Please stop me at any 
time if something is not clear. In front of you is your first ansv/er 
booklet, n^arked "3-0." First, write your sex, "male" or "female," on the 
front page of the booklet. 

Before you on the screen are tv/o faces., Faces can vary across a number 
of dimensions - such as texture of the hair, size of the lips, denseness 
of the eyebrows and so on - and it is possible to classify faces by 
describing these characteristics. 

Let's supr )se that the Larth has been invaded by creatures from another 
planet.; ^"urther, suppose tn^t these creatures looked exrictly like human 
beings, "therefore, it would be dlniost rnpossible to distinguish these 
creatures from earthlings should you happen to meet one on the street. 
Howovpr, ^h-^-.o croatiirr-^ do have an interestina foaturp--all or th^ni hnvo 
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one identical facial characteri'stiCv Your task is to discover which facial 
characteristic these creatures have in common. Looking at the two faces on 
the screen I will tell you now that onj^ of .these faces is that of a creature 
from outer space and one of his facial features is common to all of these 
creatures. It could be any one of the following; 

ALL CREATURES HAVE STRAIGHT HAIR OR 
ALL CREATURES HAVE CURLY HAIR OR 
ALL CREATURES HAVE BUSHY EYEBROWS OR 
ALL CREATURES HAVE SPARSE EYEBROWS OR 
ALL CREATURES HAVE FULL LIPS OR 
ALL CREATURES HAVE THIN LIPS 

ANY ONE of these six characteristics could be the one characteristic 
that all the alien creatures have in common.; You must determine which of 
the six charactjri sties is the correct one. 

In order to discover the correct facial feature, we will present you 
with a series of five slides showing pairs of faces similar to the pair before 
you now. Using your first answer booklet, 3-0, turn to the first page 
containing problem A and problem B. Each problem has five trials. Under 
problem A, trial numbers are listed on the left hand side and two circles 
are present for each trial - labeled left and right. On each slide or 
trial you will check the corresponding left or right circle to indicate 
which face you believe belongs to the creature from another planet. After 
you have checked the appropriate circle, you will raise your pencil to show 
me you are ready. After everyone has responded, we will remove the slide 
and tell you which one was the creature's face. Then we will present the 
slide for trial «2 and you will again check the circle corresponding to the 
face which you believe belongs to the creature, and raise your pencil.. Again 
we will tell you which ^ide actually was the creature, and so on. After 
everyone has marked his choice for trial ^5, raised his pencil and feedback 
has been given, you will wi ' te down what one single facial characteristic 
you think all the creatures possess, in the ^pace provided at the bottom. 

Let's review the characteristics again. It could be STRAIGHT hair 
or CURLY hair or i^USHY oypbrows or SPARSF eyebrow, or FULL lips or THIN lips.: 
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One and only one of these is the characteristic that all -creatures J^ave 
in common. Remember after each slide is presented, you will check the circle 
corresponding to the face which you believe belongs to the creature, raise 
your pencil to indicate you have responded, and, after everyone has responded, 
we will remove the slide, tell you which was the creature's face and then 
present the next slide. Are there any questions? All right, let's try a 
problem. Notice the trial number is shown in the middle of each slide. This 
will help you to mark your choice for each trial in the correct answer 
space. Okay, mark your first choice and raise your pencil. 

(At the end of problem A) That's the end of this problem. Don't 
forget to write what you think the solution was - what one, single character- 
istic was common to all the creatu^^es. Is everybody finished? Okay, the 
solution was straight hair, Everytime you picked the face with straight 
ha^ir your response was correct. 

Let's take a look at problem B. Notice how the facial features are 
switched around in this problem. We have again chosen one particular 
characteristic as the solution to the problem, that is, one feature which 
all "creatures" have in common. Your task is the same - you have five 
trials to determine which characteristic all creatures have in common. Re- 
member, it could be FULL lips or THIN lips or BUSHY eyebrows or SPARSE 
eyebrows or STRAIGHT hair or CURLY hair. All right, let's try problem B, . . 
Mark your choice and raise your pencil. . . That's the end of problem B, 
Don't forget to write what you think the solution was, , .Is everyone 
finished? Okay, the solution was full lips. The rest of the problems will 
continue in much the same way--we will think of a feature and you will have 
five trials to figure out what it is.. Your job is to figure out what one 
feature we have in nind on each problem Any questions? 

While the rest of the problems will be similar, the procedure we will 
follow will be a little different. Since we are interested in learning how 
people go about solving the problems, we are going to use a device to "get 
inside your heads"; that is, to find out what you are thinking as you solve 
the problems. Please turn to the next page of your answer booklet. On this 
page, labeled "Trial -1," you can see space for recording a "choice response" 
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for the first slide for problem 1. If you turn over the two pages glued 
together you will see a space for recording your choice response for 
the second slide, or trial 2. This pattern continues throughout the rest 
of the book. Turn 2 more glued-together pages and you'll find trial 3, 
and so on. Thus, every other page provides space for recording your choice 
response for the -five trials of problem 1 and similarly for problem 2. 
These pages will be referred to as choice response pages. You probably 
have noticed that another kind of page has been inserted between each choice 
response page. Turn back to the trial 2 choice response page. The facing 
page is what we will refer to as the probe page, for it is the information 
you provide on this kind of page that will enable us to probe the way in which 
you solve the problems. Notice in the column for problem 1 there is a list 
(in alphabetical order) of the possible solutions we have gone over before-- 
BUSHY eyebrows, CURLY hair, FULL lips, SPARSE eyebrows, STRAIGHT hair, and 
THIN lips. Next to each possible solution is a space for you to make a check 
mark. On this page you will show us what you are thinking, by checking 
those solutions you think couJd_sjtiiU)e_c^ For example, if you 
think the solution could be either bushy eyebrows or curly hair, you would 
check both bushx and curly. Is this clear? ^^.e. We'll go through problem 1 
like this. 

We'll show you the first slide, and you check the side you think is 
correct on the Trial 1 chojce response page. After you have marked your 
choice, you will raise your pencil. When everyone has responded and 
raised his pencil, we will remove the slide and tell you which side was 
correct. You will then turn over two pages to the Trial 1 ^Lohe page, and 
put a check by any of the facial characteristics you think could still 
be thfe solution to the problem. After you have finished, foil the pages 
on the left under the rest of the book. Make sure you fold the probe page 
over after you have marked it. When everyone hc:s finished, we'll show you 
the second slide. You'll mark your choice on the trial 2 choice response 
page, and raise your pencil. We will remove the second slide and tell you 
which side was G(^rrect. You will then turn two pages to the trial 2 probe 
page and mark tho.e features you still think could be the solution. We con- 
tinue like this through trial 5. At the bottom of thr trial 5 probe page 
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there is a space for you to write down the one characteristic, you think is 
the solution to the problem. If you're net sure, give your best guess. 

Reniembt- to mark on each probe page: BUSHY, if you think bushy eyebrows 
could still be the soUtion, CURLY, if you think curly hair could still 
be the solution, FULL, if you think full lips could still be the solution, 
SPARSE, if you think spars? eyebrows could still be the solution, STRAIGHT, 
if you think straight hair could still be the solution, and THIN, if you think 
thin lips could still be the solution. These are listed on the poster on 
the front wall to help you remember. You may check as many characteristics 
as you like on the probe pages, but only write one characteristic in the solu- 
tion box at the end of each problem. 

Turn back now to the trial fl choice response page. Fo- Problem #1, 
notice the facial features again have been moved around. Let's begin 
the problem.. Check the corresponding left or right circle to indicate 
which face you believe is correct. Raise your pencil when you have done 

this. The correct side is . Turn the choice response page over. On 

the trial fl^ probe page, check the characteristics you think could still 
■e the solution. When you're finished, fold the pages on the left underneath. 

Turn to the trial 2 choice page. Check the corresponding left or right 
circl€ to indicate which face you believe belongs to the creature. Raise 

your pencil when you have done this. The correct side is . Turn the 

choice response page over. On the trial 2 probe page check the characteris- 
tics you think could still be the solution. When you finish, fold the pages 
underneath. 'We'll continue just like this through trial 5. As soon as you have 
completed the trial 5 probe page and written your solution, turn back to the 
trial 1 choice response page, and raise your pencil. The solution to problem 1 
was sparse eyebrows. Whenever you picked the face with sparse eyebrows you 
were correct. Any questions? 

Is everybody on the trial 1 choice response page for problem 2? 
Here is a slide of two new faces in an even more scrambled order. Notir 
the features have not changed.. The faces still have CURLY or STRAIGHT 
hair, or BUSHY or SPARSE eyebrows, or FULL or THIN lips, but they are jumbled 
around. Your task is the sane a .o - to identify the one character! :.t ic that 
I have in mind, that is the solution to the problem. Let's begin. . .. 
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(Prompting as necessary.) Okay, the solution is thin lips. Put your 3-0 
booklet under the brown file and take the 4-1 booklet out of the file. 

All right, open your 4-1 booklet to the trial 1 choice re sponse page. 
All booklets will now list six problems on each choice response page, and 
the six corresponding problems on each probe page. 

The first pair of faces you see on the screen has another possible facial 
dimension that you must consider - eyes.^ The. po:,5ible solutions now include 
WIDE eyes or SQUINTY eyes. There are now eight possible solutions to each 
problem. Turn to the probe page for trial 1. The possible solutions are 
STRAIGHT hair or CURLY hair, or BUSHY eyebrows or SPARSE eyebrows, or FULL 
lips or THIN lips, or WIDE eyes or SQUINTY eyes. Notice all these solutions 
are in the probe page in alphabetical order,. 

Ready for problem #1 . .(Prompting only if necessary). . .The solution 
is curly hair. Before beginning the next problem, we want to remind you that 
the facial features may also be scrambled like this. 'Notice the facial 
features are moved around but once again the characteristics have not changed. 
They are BUSHY eyebrows or SPARSE eyebrows, or FULL lips or THIN lips, or 
STRAIGHT hair or CURLY hair, or WIDE eyes or SQUINTY eyes. Remember, one and 
only one feature is the solution to each problem.. Remember also that fea- 
tures could be scrambled like this. Any questions? Okay. Please don't 
make any notations in your booklets other than checkmarks in the circles 
provided., Also, please do not turn back any pages through your -)ok, except 
when we're all ready to begin a new problem. (Go through all four-dimension 
problems. 

(After all four-dimension problems):; 

We'll now take a short break. Please do not discuss the problems during 
the break.^ 

(After break); 

All right, open your 6-1 booklet to trial 1 choice response page. 
The pairs of faces are slightly more complex now. Iwo more possible facial 
dimensions must be considered - glasses and noses. The solution could be 
either round or square glasses. Also, it could be either a long or a short 
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nose. So there are now 12 possible solutions to the problem. Turn to 
the trial 1 probe page. The solution could be STRAIGHT hair or CURLY hair, 
or BUSHY eyebrows or SPARSE eyebrows, or FULL lips or THIN lips, or WIDE 
eyes or SQUINTY eyes, or ROUND glasses or SQUARE glasses, or LONG nose or 
SHORT nose. Remember, one and only one feature is the solution to any one 
problem. 

Ready for problem 1. . .The solution is long nose^ Before we begin 
the next problem, I want to remind you that the facial features may also be 
scrambled like this. Notice the facial features are moved around again; 
however, the characteristics are the same— STRAIGHT hair or CURLY hair, or 
BUSHY eyebrows or SPARSE eyebrows, or FULL lips or THIN lips, or WIDE eyes 
or SQUINTY eyes, or ROUND glasses or SQUARE glasses, or LONG nose or SHORT 
nose. Remember also that features could be scrambled like this. Any 
questions? Okay, let's begin problem 2. . .. 

(After all six-dimension problems); 

We'll now take a short break. Please do not discuss the problems during 
':he break. 

(After break) : 

All right, open your 8-1 booklet to trial 1 choice response page. On 
the next pair of faces, two new possible facial dimensions have been added - 
facial hair and scar. The solution now could be either a mustache or a 
beard or scar on the right side of the face or on the left side of the face. 
Determine the side of the scar as you view it. 

So there are now 16 possible solutions to the problem. Turn to the 
trial 1 probe page. The solution could be STRAIGHT hair or CURLY hair, 
BUSHY eyebrows or SPARSE eyebrows, FULL lips or THIN lips, WIDE eyes or 
SQUINTY eyes, ROUND glasses or SQUARE glasses, LONG nose or SHORT nose, 
BEARD or MUSTACHE, RIGHT or LEFT SCAR. Remember, one and only one feature 
is the solution to each problem. 

Ready for problem 1?. . .The solution is round glasses. Before we 
continue, remember that the facial features may be scrambled like this. 
Notice the facial features are switched again. However, the chara<iteristics 
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are the same-STRAIGHT hair or CURLY hair, BUSHY eyebrows or SPARSE eyebrows, 
FULL lips or THIN lips, WIDE eyes or SQUINTY eyes, ROUND glasses or SQUARE 
glasses, LONG nose or SHORT nose, BEARD or MUSTACHE, RIGHT or LER SCAR. 
Remember, they could appear like this, too; the scar, however, will always 
be on the left or right. Let's begin problem 2. . .(continue with remaining 
eight-dimension problems)." 
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